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ABSTRACT 

The study involving 65 school psychologists, 38 
special education teachers, and 21 "naive" university students (those 
not in education or psychology related programs) examined the extent 
to which school personnel were able to differentiate learning 
disabled (LD) students from low achievers by examining students' 
patterns of scores on psychometric measures. Each S was provided with 
forms containing information 6n 41 test or subtest scores of 9 school 
identified LD and 9 nonLD fourth grade students and was instructed to 
indicate' which he/she believed were LD and which were nonLD. Accuracy 
and interjudge agreement confirmed the difficulty of differentially 
diagnosing students with learning disabilities and suggested that 
havrng information from a battery of tests does not increase the 
decision maker's ability to discriminate between LD and nonLD 
students* It was recommended that current assessment practices be 
simplified. (Author/SBi 
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Abstract 

This study examined the extent to which school personnel were 
able to differentiate learning disabled students from low achievers 
by examining students' patterns of scores on psychometric measures. 
Subjects were 65 school psychologists, 38 special educatioii teachers, 
and a ''naive'* group of 21 university students in programs unrelated 
to education or psychology. Each ji^dge was provided with forms con- 
taining information on 41 test or subtest scores of 9 school- 
identified LlD students and 9 non-LD stuoents and instructed to 
indicate which he/she believed were learning disabled and which 
were non~learning disabled. Accuracy and inter-judge agreement 
confirmed the difficulty of differentially diagnosing students with 
learning disabilities and suggested that^baving information from a 
battery of tests does not increase the decision maker's ability to 
discriminate between LD and non-LD students. ConsiJering the extreme 
difficulty in integrating empirical and non-empirical information, it 
is recommended that current assessment practices be simplified. 
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Differentiating LD and Non-LD Students 
''1 Know One Whien I See One" 



Learning disabilities researchBrs typically have attempted to iden- 
tify the salient characteristics that distinguish LD students from other 
students, especially those identified as "educable mentally retarded," 
"emotionally disturbed," or "normal" (Ackerman, Peters, & Dykman> 1971; 
Bussell, Huls, & Long, 1975; Gajar, 1979; Wagonseller, 1973; Webster & 
Schenck, 1978). A major difficulty with such studies is that they com- 
pare learning disabled students to populations with obviously different 
characteristics. In comparison to the mentally retarded, intellectual 
differences are expected and observed; compared to students classified 
as emotionally disturbed, differences in the number of behavior problems 
are expected and observed; in comparison to "normal" students, differ- 
ences in the level of academic achievement are expected and observed. 
Such investigations provide little new or useful information. 

Difficulties typically arise when diagnostic personnel attempt to 
differentiate LD students from those who are simply slow learners. 
These students are ones who fail to acquire academic skills and fail 
to meet the objectives that the school sets for them. A critical diag- 
nostic problem is seen in efforts to decide, from among the population 
of students who are not "making it" in school, who should be declared 
LD and provided with special educational interventions. Research is 
nearly nonexistent on this critical diagnostic problem. 

Ysseldyke, Aigozzine, Shinn\ and McGue (in press) found a relatively 
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large number of fourth-grade students who were low achieving but 
not labeled as LD, despite the fact that school personnel believed 
that few such students existed in their schools. The students scored 
in the lowest quartile on a group achievement test, but had not been 
identified as LD by their school districts. 

Given that the two groups of students (low achievers and LD) do 
exist, it becomes important to investigate the basis for distinguishing 
them. Ysseldyke et al. (in press) examined the extent to which measures 
of cognitive ability, academic achievement, perceptual-motor skills, 
self-concept, and behavior problems discriminated between low achievers 
and school-identified LD children. Their results indicated considerable 
similarities between the two groups; an average of 96% of their scores 
were within a common range and the performance of the LD and iow-achif»ving 
students on many of the subtests was identical. Based on individual 
psychometric measures, Ysseldyke et al. raised serious concerns 
regarding the differential classification of low-achieving students as 
either learning disabled or non-learning disabled. 

Some school personnel have argued, however, that decisions actually 
are made on the basis of the pattern of scores that students obtain on 
several different psychometric instruments. It thus becomes important 
to ascertain the extent to which school .'ecision makers con distinguish 
between the two groups when given patterns of scores on multiple measures. 

The use of a number of scores to identify learning disabled students 
is, in fact, mandated by the U,S, Office of Education, I'he August 2'^, 



1977 Federal Register specifically states that ''no single procedure is 
used as the sole criterion for determining an appropriate educational 
program for a child'' (p. 4249?). To determine the existence of a speci- 
fic learning disability, "a severe discrepancy between achievement and 
intellectual ability" ( Federal Register , 1976, p. 65083) must be demon- 
strated. Both of these federal guidelines necessitate the use of more 
than one test in an evaluation of a student, a procedure which, in turn, 
requires an examinat4.on of a student's pattern of scores. 

Although an ability-achievement discrepancy is a typical requirement 
for classification as learning disabled, by no means is there consensus 
that it is a useful concept. Salvia and Ysseldyke (1981) caution that 
before a student's difference scores can be interpreted, it must be 
established that the differences are reliable; differences in stardard- 
ization samples for the two tests also should be considered. In practice, 
these cautions frequently aie ignored. In addition, a number of studies 
(Algozzine, Forgnone, Mercer, & Trifiletti, 1979; Algozzine. Ysseldyke, 
& Shinh, 1980; ysseldyke & Algozzine, 1979) argue that learning disabili- 
ties is largely a category of underachievement , thus suggesting that 
the degree of students' achievement deficits also should be considered 
whe. determining eligibility for LD placement. 

To investigate the extent to which judges could differentiate between 
non LD students and those who are learning disabled, Epps, McGue, and 
Ysseldyke (1981) presented IS judges with an army of lest information sim- 
ilar to that typically presented in placement team meetings and asked ihem to 
determine which of 99 students were LD and which were not. The judges were 
individuals who had previous experience in psychoeducat ional assessment 
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and/or placement team decision making^ plus experience in research on 
learning disabilities. Epps et al. (1981) found that the judges were 
extremely inaccurate and rarely in agreement with each other. The 
judges also emphasized different factors in making their decisions. 
As Epps et al. noted, however, the judges were not working within the 
educational system and were not making day-to-day decisions about the 
classification of students. To overcome this limitatio'^, the present 
study was designed so that actual decision makers in the field could 
attempt to use patterns in the data to differentiate LD and non-LD 
students. In addition, a comparison group of individuals unfamiliar 
with psvchoeducational do.cision making was included to provide base- 
line accuracy information. 

Method 

Subjects 

The sample consisted of 65 school psychologists, 38 special educa- 
tion teachers (SLD or SLBP), and 21 university students in programs un- 
related to education or psychology (the "naive" group) . The school 
psychologists were randomly selected from the 1979 NASP Directory . 
Since there is no comparable national directory for special education 
teachers, their participation was solicited in two ways. First, the 
names of 175 elementary schools were randomly selected from a directory 
of U.S. schools. The packet of materials was sent to the "LD Teacher" 
at eich of these schools. Second, the name of one director of special 
education was randomly selected from each state and from the District 
of Colfabia. Each director was sent two packets of materials and re- 
quested to give them to "any two SLD/SLBP teachers from [his/her] 
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district who would be willing to provide input for our LD/Non-LD clas- 
sification study." Both the names of the elementary schools and the 
special education directors were obtained from the School Universe 
D aca Book, School Year 1976-1977 . 

Altogether, a total of 306 forms was sent out to school psychologists; 
277 were sent out to elementary schools or to special education directors 
for the special education teachers. For participating in the study, each 
subject had the opportunity to earn a cash prize of $25, $15, or $10. 
The relatively low response rates for school psychologists and teachers 
(21.2% and 13.7%, respectively) seemed to be dye in part to the difficulty 
in reaching the individuals (incorrect addresses, etc.) and to the time- 
consuming nature of the task. 

The participation of the naive group was solicited from two under- 
graduate courses, one in mechanical engineering and one in business 
management. The main criterion for their inclusion in the study was 
that they were inexperienced in psychoeducational assessment; that is, 
they had not taken education courses nor any psychology classes beyond 
introductory psychology. For participation in the study, each subject 
earned $5. 
Mato.rials 

Forms containing the chronological age and sex of 18 fourth-graders 
and information on their actual test scores in five domains were pre— 
p.'^red. These domains and the tests used were: (a) aptitude (WISC-R, 
Woodcock-Johnson Tests of Cognitive Ability, (b) academic achievement 
(Woodcock-Johnson Tests of Achievement, PIAT, Stanford Achievement 
Test - math calculation and math concepts), (c) perceptual-motor 
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(Bender, VMI), (d) self-concept (Piers-Harris), and (e) behavior 
probl ems (Pe ter son— Quay ) • Across t hesc five doma ins , test scores 
were provided on 41 variables. 

Based upon the school^classif ication and low-achievement criteria, 
the 18 cases were selected from a larger study that compared the test 
performance of 50 school-identified LD students to that of A9 low- 
achieving students who had not been identified as learning disabled. 
Nine of the 18 cases were s»-hool-identif ied LD students and nine were 
low achievers from melropolitan Minneapolis and St. Paul schools. 
Only eight of the low achievers were included in the analysis.^ The 
LD/non-LD classification was not indicated on the judges' test score 
forms. The exact criteria for identification of students as LD by the 
schools were unknown. The low-achieving group had not been identified 
as LD by their school districts, but scored at or below the 25th per- 
centile on the Iowa Tests of Basic Skills which had been administered 
by the schools during the fall of the school year. 

As a criterion for considering a student LD, the degree of 
students* achievement deficits was taken into account and contributed 
to the case selection procedure. The achievement deficits were oper- 
ationalized in two ways using subjects' observed test scores in written 
expression, basic reading skill, reading comprehension, mathematics 
calculation, and mathematics reasoning. In one condition (1.0 SD) , 
the student was considered LD by definition if the score earned on 
at least one of the criterion measures was between 1.0 and 1.5 stan- 
dard deviations below the mean (standard scores of 73 to 85). In the 
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other condition (1.5 SD) , the student was considered LD if the score 
earned was at least 1.5 standard deviations below the mean (standard 
scores of 77 or below). Tables were developed listing school-identif ied 
LD students and low achievers who met the criterion of LD or non-LD ac- 
cording to the low achievement definition. Cases were then randomly selec- 
ted from the categories, resulting in the numbers presented in Table 1. 



Insert Table 1 about here 
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Along with the schools' classification and the two operationaljzat ions 
of the low achievement definition, the federal definition of learning 
disabilities, taken from the December 29, 1977 Federal Register , was 
used to classify students as LD or non-LD. That definition lists . 
seven areas in which a child may be determined to have a specific learn- 
ing disability: oral expression, basic reading skill, reading compre- 
hension, mathematics calculation, mathematics reasoning, listening com- 
prehension, and written expression. However, the federal definition 
does not specify the amount of discrepancy between ability and achievement 
that is required. In the present study, the federal definition was oper- 
ationalized 1:. tw^ ways using subjects' observed aptitude-achievement 
test scores in the areas of intelligence and achievement. The following 
measures were used to ascertain discrepancy in the seven areas: 
(a) written expression (UISC-R Full - W-J Written Language Achieve- 
ment) , (b) basic readii.g skill (WISC-R Full - W-J Reading Achieve- 
ment) , (c) reading comprehension (WISC-R Full - PIAT Reading Com- 
prehension), (d) mathematics calculation (WISC-R Full - Stanford 
Mathematics Calculation), and (e) mathematics reasoning (WISC-R 
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Concepts). In one condition (1.0 SD) , the student was considered LD 
by definition if there was between a 1.0 and 1.5 standard deviation 
difference on at least one of the five aptitude-achievement discrepancy 
scores (difference scores of 15 to 22). In the other condition (1.5 
SD) , the student was considered LD if there was at least a 1.5 standard 
deviation difference on at least one of the five aptitude-achievement 
discrepancy scores (difference scores of 23 or more). Table 2 indicates 
the ntunber of cases of school-identified LD students and low achievers 
who were classified as LD and non-LD based on the school's definition 
and the federal definition. 

Insert Table 2 about here 

Along with the "test scores" form, the packet of materials given 
to judges included a "tests administered" form that specified which 
tests were used in each of the five areas. A third sheet provided 
normative data on those measures that did not have a mean of 100 and 
standard deviation of 15. These included the Bender Visual-Motor 
Gestalt Test, the Developmental Test of Visual-Motor Integration, and 
the Peterson-Quay Behavior Problem Checklist. Also provided were a 
full group mean and standard deviation for the Piers-Harris Children's 
Self-Concept Scale, based upon the larger sample of 99 students. (See 
the Appendix for copies of all materia. 3 provided to judges.) 
Procedure 

Each judge received the packet of materials and a response form 
and then was instructed to examine the test scores for the 17 students 
and to indicate which he/she believed were learning disabled and which 



were non-learning disabled. After a decision had been made regarding 
each child's classification, the judge indicated his/her degree of 
certainty on a ^TcDur-point scale from very uncertain to very certain. 
Then each judge rated the extent to which different factors affected 
his/her decisions. 

Results 

Judges ' Accuracy 

Judges' accuracy in identifying the non-LD and school-identified 
learning disabled students is reported in Table 3. A comparison was made 
between judges' ratings and school classification, between judges' ratings 
and the classifications from each of the two levels of achievement deficit, 
and between judges' ratings and the classifications from each of the two 
federal definitions. The accuracy percentages were computed separately 
for LD and non-LD students within each of the five conditions and F 
values were calculated. All three group? of j 'ges were in agreement 
with the schools' classifications only about half of the time and there 
was no significant difference between the teachers, the school psycholo- 
gists, and the naive group in their overall agreement with the schools 
[F(2,121) = .AA, £ = .6A] . 

When the 1,0 SD achievement deficit was considered, whereby a 
student was classified as LD if the score earned on at least one of 
the criterion measures ;^as between 78 and 85, again there was no 
significant difference between the three groups in their overall 
accuracy [F(2,121) = ,85, £= ,43]. However, when the 1,5 achieve- 
ment deficit t'as considered, whereby a student was classified as LD 
if the score earned on at least one of the criterion measures was 77 
or below, there was a significant di^f^ence IF(2,121) = 18,29, 



p < *0001. The naive group was more accurate in identifying those 
students who had scores of 77 or below (73»1%) than were either the 
teachers (54.7%) or the school psychologists (56,5%). 

. Insert Table 3 about here 

Although che niive group was more accurate in identifying those 
students who were classified as LD base4 upon the 1.5 SD low achievement 
definition, they were less accurate (26 •3%) in identifying those students 
who had a moderate aptitude-achievement discrepancy (1*0 SD federal defin 
tion) then were either the teachers (43.7%) or the school psychologists 
(42.4%) • The naive group was also less accurate (35.0%) in identifying 
those students who had a severe aptitude-achievement discrepancy (1.5 SD 
federal definition) than were either the teachers (53.6%) or the school 
psychologists (53.6%). However, the teachers and school psyc^plogists 
were in agreement with the two operationalizations cf the federal defini- 
tion for only about half of the students. 
Judges' Leniency 

The second factor that was examined was lepiency, which reflects 
the percentage of children classified as LD. By definition, a lenient 
judge classifies more students as LD than as non-LD. A non-lenient 
judge classifies fewer students as LD than as non-LP. As can be seen 
in Table 4, che overall leniency of the three groups did not differ sig- 
nificantly [F(2,121) = 1-2, £= .32). In other words, teachers (39.1%), 
schjol psychologists (35.6%), and the naive group (32.5%) identified an 
approximately equal proportion of the students as LD. Overall, the 
three groups were less lenient in their cttssi f icntions than 
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were the schools and identified fewer students as LD than the 50% 
which the schools had so identified. 



Insert Table 4 about here 

As can be seen in Table 4, students 1, 12, and 17 met the criteria 
for LD based upon the school definition and the two low achievement 
definitions. Yet, fot student 1, only 26.3% of the teachers and 18.5% 
of the school psychologists identified him as LD. In contrast, 85.7% 
of the naive group did so. Despite the fact that 73.7% of the teachers 
and 81.5% of the school psychologists incorrectly classified the student, 
regardless of which definition was used as a criterion, as a group each 
was certain of its decision (average degree of uncertainty: 2.97 for 
teachers and 3.05 for school psychologists on a scale where 2 = uncertain 
and 3 certain). For student 12, greater percentages of teachers (44.7%) 
and of school psychologists (58.5%) dentified him as LD than for student 
1. Still a significantly greater percentage of the naive group identi- 
fied him as LD (90.5%). 

Students 5, 6, 10, 14, and 17 met the criteria for LD based upon 
the school definition and the two federal definitions. For three of 
these four students (6, 10, and lA), significantly greater percentages 
of teachers and school psychologists identified them as LD than did 
the naive group. For student 5, however, there was no significant dif- 
ference in the percentages of judges classifying him as LD, 

There was only one student (student 17) who met the criteria for 
LD based upon all five conditions, namely, the school definition, the 



two low achievement definitions, and the two federal definitions. 
For student 17, a greater percentage of the naive group (81.0%) iden- 
tified him as LD than did the teachers (63.2%) or school psychologists 
(64*6?), although this difference was not significant* Despite the 
fact that this student was classified as LD based upon all five defini- 
tions, a considerable percentage of teachers (36.8%) and schoui 
psychologists (35.4%) classified him as non-LD. In general, for 
all three students (1, 12, and 17) who met the criteria for definition 
as LD under the school and low achievement definitions, a greater per- 
centage of the naive group identified the students as LD^than did the 
teachers or school psychologists. The reverse was true when the school 
and federal definitions were considered. For three of th^se four 
students (6, 10, and 14), greater p-ercentages of the teachers and school 
psychologists identified the students as LD than did the naive group. 

Table A also provides information on two students (8 and 13) who 
did not meet the criteria for the school definition or for the two low 
achievement definitions of LD. That#is, the schools did not classify 
them ^s LD nor were any of their scores on standardized tests 85 or 
below. Yet, a fourth of the teachers (26.3%) and 13.8% of the school 
psychologists, along with 38.1% of the naive group, classified student 
8 as LD. Similarly, a substantial number of teachers (18.4%) and school 
psychologists (12. 3%), along with 14.3% of the naive group, also classified 
student 13 as LD. Another student (student 11) did not meet the cri- 
teria for the school definition or for the two federal definitions of 
LD. That is, the schools did not classify him as LD nor were any of 
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his ability-achievement discrepancy scores as great as 15 points. 
Yet 34.2% of the teachers and 23.4% of, the school psychologi sts» along 
with 66.7% of the naive group, classified student 11 as LD. 
Intergroup Agreement ' 

The third factot that was investigated was intergroup agreement, 
the extent to which Judges of one group agreed with judges of another 
group in their ratings of the students. For each of the three experi- 
mental groups, teachers, school psychologists, and naive, the proportion 
of group members who classified each of the J 7 ^as-^.s as LD was determined. 
The 17 proportions represent the extent to which teacjiers. for example, 
in general, identified each of the students as being learning disabled. 
In deriving tTie index, Che' point biserial correlation between each 
judge's ratings and the thre?. group composites was then calculated as a 
measure of the extent to which each judge agreed wiXh teachers, school 
psychologists, and naive. 

Considering that the naive group had no training in psychoeducational 

decision making, we would expect them to be in low agreement with school 

personnel. As can be seen in Table 5, the naive, group was in low agree- 

I 
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ment with both the teachers' composite group decisions (.04) and the 
school psychologists' composite group decisions (.08). For professionals 
trained In this decision-making process, we would expect to find high 
agreement. Teachers were found to agree as much with the school psycholo- 
gists' composite (.50) as with their ^.wn (.49). Similarly, school 
psychologists agreed as much with the teachers' composite (.47) as with 
their own (.53). Thus, high agreement was observed between the two pro- 
fessional groups. 
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Insert Table 5 about here 



FaccoTS Influencing Decisions / 

The'fQurth factor that was examined was the degree to which certain 
factors influenced judges' decisions. Each judge rated the effect that 
the factors had on her/his decisions* As can be seen in Table 6, there 
was a high degree of similarity in the ratings across the three groups • 
There were no significant differences between the three groups on the 
rated importance of intellectual measures, measures of academic achieve- 
ment, perceptual-motor tests, adaptive behavior, personality test data, 
and behavioral recordings. Both teachers and school Psychologists 
rated intellectual factors, academic achievement, and discrepancies 
between expected and .actual achievement very highly (range of ratings: 
4.2 to 4.4). In contrast, the child's sex and personality were rated 
as low in importance. 



Insert .Table 6 about here 
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In their ratings of the importance of various factors on their 
decisions, judges were likely to have been guided by their knowledge of 
the definitions for learning disabilities. Thus, Iheli ratings might 
reflect more of what they believe were important faccors than of what 
factors actually were important in their decision-nuking process. To 
e^tamine the variables that the three.groups of judges actually used to 
classify the students, mean difference scores were examined. For e.ich 
group, the means of each of the 41 original test scores were calculated 
for students whom the judges in the group classified as LD and for students 
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whom they classified as non-LD» In addition, the means for five dis- 
crepancy scores (WISC-R Verbal - WISC-R Performance, W-J Reading 
Aptitude - W-J Reading Achievement, W-J Mathematics Aptitude - W-J 
Mathematics Achievement, W-J Written Language Aptitude - W-J Written 
Language Achievement, and WISC-R Full - PIAT Total) were calculated. 
The 46 absolute mean differences were determined and then rank ordered 
from largest to smallest. For example, for a group of judges, the var- 
iable ranked first would be that variable for which the students judged 
learning disabled weie most different from the students judged non- 
learning disabled • Thus, that variable apparently played a major role 
in the differentiation of LD from non-LD for that group of judges, al- 
though not necessarily for each judge within that group. For each group 
of judges, the 11 highest-ranked variables are presented in Table 7. 

Insert Table 7 about here 

As can be seen in Table 7, there is a considerable overlap in the 
variables for teachers and school psychologists. Of the 11 variables 
listed, 9 are common to both. In addition, there is overlap in the var- 
iables for the naive group as compared with the teachers and school 
psychologists. Of the 11 variables listed for this group, k of them 
are common to those of both the .eachers and school psychologists and 
another 2 are common to those of the school psychologists. At the 
group level of analysis, average mean difference scores suggest that 
both teachers and school psychologists primarily examined achievement 
measures with less emphasis given to cognitive measures, aptitude- 
achievement discrepancy, and school-related aptitude measures. In 
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contrast, the naive group apparently gave greater consideration to 
individual subtests • Although the teachers gave a high rating to the 
degree to which intellectual measures influenced their decisions 
(rating of 4,3), the mean difference in IQ between the group of students 
they identified as non-LD versus those identified as LD was only 1/2 a 
point* Similarly, the school psychologists gave a high rating to In- 
tellectual measures (4,2), yet the mean difference in IQ between the two 
groups was only one point. However, for the naive group, who rated in- 
tellectual measures at 3,9, the mean difference in IQ between the two 
groups was 13.1 points with the non-LD group scoring higher. 

Discussion 

There is little doubt that the identification of students with 
learning disabilities is fraught with difficulties. Not only are there 
no meaningful psychometric differences between students identified by 
schools as learning disabled and those who are performing poorly in 
school and not identified as LD (Eno & Woehlke, 1980; Ysseldyke et al.> 
in press), but judges experienced in psychoeducational assessment, al- 
though not working within the educational system, also were unable to 
identify accurately the LD students when presented with their patterns 
of scores on numerous devices (Epps et al., 1981).' The results of the 
present study confirm the difficulty of differentially diagnosing stu- 
dents with learning disabilities, even when actual school decision 
makers are the judges, and suggest that having access to information 
from a battery of tests does not increase the decision maker's ability 
to disc^mlnatc between LD and non-LD students. Despite the fact 
that ihere was high intergroup <igreement between teachers and school 
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psychologists and low intergroup agreement between the naive group and 
school personnel, all three groups still identified approxlmalely the 
same number of students as LD. And despite the fact that the naive 
group was the only ZJ^^^P having no specific training in psychoeducational 
decision making, they were significantly more accurate in identifying 
those students classified as LD using the 1,5 standard deviation low 
aciiievement definition. For the student who met the criteria for 
definition as LD under all five definitions, a greater percentage of 
the naive group identified him as LD than did either the teachers or 
school psychologists. In addition, although two of the students were 
classified as non-LD in four of the five conditions, a substantial num- 
ber of both teachers and school psychologists classified them as LD. ^ 

The difficulty in discriminating the LD from the non-LD students 
is likely to have been influenced by a number of factors. First, test 
results were presented in standard scores with a mean of 100 and standard 
deviation of 15. Such a format may have been confusing for those 
teachers who are more accustomed to grade and age equivalents and for 
those school psychologists who are more accustomed to WISC-R scaled 
scores with a mean of 10 and standard deviation of 3. Second, some of 
the judges may have been unfamiliar with the tests and their school 
systems may have employed a different battery of tests. Third, a lack 
of consistent agreement among school personnel reflects, in part, the 
varying state guidelines for LD identification. Fourth, other cypes of 
information, beyond psychometric data, may contribute to LD identification 
and classification decisions. Indeed, it is desirable to have direct 
observational data on the student's behavior and actual classroom work 
and information from criterion-referenced tests. Tl:e format of this 
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study necessitated a test-centered a"»pioach to diagnosis which 
precluded consideration of qualitative information. Fifth, 
classification decisions are, in fact, not made by one person. 
Indeed, the August 23, 1977 Federal Register states that each public 
agency shall "insure that the placement decision is made by a group 
of persons" (p. 42497), And sixth, there are certain external con- 
straints that frequently are considered in classification decisions 
(e.g,, the availability of an LD program in the student^s school). 

Federal guidelines mandate the use of more than one test in an 
evaluation of a student and thus require an examination of a student's 
pattern of scores. Consequently, school decision makers must evaluate 
a multitude of empitiical information as well as non-empirical criteria. 
They nust decide what data are relevant and valid and then weigh the 
importance of various pieces of data and combine or organize them in 
some manner. This approach to evaluation using multiple sources of 
information is nonproblematic if we assume that individuals are capable 
of conducting such complex tasks. However, a number of studies have 
consistently demonstrated that we are far less capable of making com- 
plex judgments than we typically believe. For example, Oskamp (1965) 
demonstrated the low accuracy of judgments of trained psychologists when 
presented with a complex judgment task. As the psychologists gained 
further information about the case, they became increasingly confident 
in their decisions, although their average final accuracy never exceeded 
28%. Indeed, increasing feelings of confidence f*s more information is 
gathered is not a sure sign of increasing accuracy in the decision. 

Why we experience difficulty in making complex judgments, such 
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as identifying LD students, is uncertain. First, it may be in part 
due to the insufficient and inaccurate feedback that we often receive 
about our judgments. In other words, there is wide disagreement about 
what constitutes the category of learning disabilities and no specific 
external criterion with which we can check our decisions. Second, in- 
dividuals frequently do not adhere^ to certain basic normative rules 
when foiming judgments. For example, they may fail to consider the 
technical adequacy of various psychoeducational devices or the appro- 
priateness of using various tests with certain popi'lations . And third, 
poor judgments may be caused by "cognitive overload." In many cases, 
information becomes too complex for us to process optimally. In attempt- 
ing to use simplifying judgment strategies, we are likely to ignore 
much information and often will integrate other information inaccurately. 
The hum;=:a tendency to simplify complex tasks appears to be most commonly 
expressed by using new information to confirm an existing hypothesis 
rather than disregarding the new information (Jason, 1978). Thus, in 
the present study, the decision makers were likely to have evaluated 
the extensive array of data in terms of its contribution to their ini- 
tial hypothesis of a student^s classification. 

When psychoeducational decisions are impaired by bias in assessment 
or failure to consider certain normative principles, training specifically 
aimed at remediating these problems may improve decision making. How- 
ever, when decisions are impaired by cognitive overload, corrective 
training may provide little or no improvement. 

Considerable time and expense are expended by school personnel to 
determine the classification of students referred for evaluation. This 
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process of evaluation typically involves the use of numerous psycho- 
educational devices which frequently requires the services of several 
diagnostic personnel. Considering the extreme difficulty in inte- 
grating the multitude of empirical and non-empirical information^ 
and given the fact that there are no meaningful psychometri<f differences 
between school identified LD students and low achievers .no are not 
identified as LD, a change is necessary in current assessment practices. 
Such a modification would desirably simpl if y_,the^ evaluation process so 
that fewer tests are used. In addition, the placement of a student in 
an educational setting might be determined more profitably by examining/ 
his/her ^^sent repertoire of behaviors rather than by collecting a 
multitude of ^st scores and then placing the student according to the 
diagnostic label (Deno & Mirkin, 1977; Hallahan & Kauffman, 1977; 
Neisworth & Greer, 1975). 

Given current definitions of the category of ^'learning disabilities" 
and the extreme heterogeneity of the population of students considered 
learning disabled, there is little basis for the hope that school person- 
nel are going to be able to diagnose LD students reliably. Indeed, the 
educational utility of the category of "learning disabilities" is 
suspect . 
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Footnote 

'''One of the low achievers was deleted from the study due to typo- 
graphical errors that were detected in several subtest scores after 
materials had been mailed to teachers and school psychologists. 
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Table 1 

Number of Students Within Categories of School and 
Low Acnlevement Definitions of Learning Disabilities 



T A U4 £j J ^ School Definition 

Low Achievement Definition 



LD Non-LD 



Non-LD 


(86 


or 


above) 


3 


2 


1.0 SD LD 


(78 


to 


85) 


3 


3 


1.5 SD LD 


(77 


or 


below) 


3 


3 



lumbers in parentheses indicate the achievement ueficit scores (in standard 
score units: X = 100, SD = 15) required for each oporationalization of the 
low achievement definition. 
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Table 2 

Number of Students Within Categories of School 
and Federal Definitions of Learning Disabilities 



_ , t rj SI School Definition 

Federal Definition 



LD Non-LD 



Non-LD 




(1-14> 


1 


1 


1.0 SD 


LD 


(15-22) 


3 


2 


1.5 SD 


LD 


(23 or more) 


5 


5 



lumbers in parentheses indicate the ability-achievement discrepancy 
scores required for each operationalization of the federal definition. 
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Table 3 

Judges' Accuracy in Identifying LD and Non-LD Students 
According to Five Definitions 







Teachers 


Sch. Psyc. 


Naive 


F(2,121) 


P 


School Definition 












% Correct LD 


42.8 


41.4 


38.1 


.39 


.68 


% Correct Non-LD 


65.1 


70.1 


73.8 


1.76 


.18 


% Overall Accuracy 


53. 3 


55. 2 


54.9 


.44 


.64 


1.0 SD Achievement Deficit 












% Correct LD 


45.6 


43.^ 


38.5 


1.08 


.34 


% Correct Non-LD 


76. 7 


82.5 


81,9 


.93 


.40 


% Overall Accuracy 


54. 7 


54.8 


51.3 


. 85 


. 43 


1.5 SD Achievement Deficit 












% Correct LD 


41.2 


38.9 


57.9 


5 79 


004 


% Correct Non-LD 


62.1 


66.1 


81.4 


7.68 


.0007 


% Overall Accuracy 


54.7 


56.5 


73.1 


18.29 


.0000 


1 .0 SD Federal Definition 












% Correct LD 


40.3 


37.6 


26.7 


4.39 


.014 


% Correct Non-LD 


69.7 


79.2 


23.8 


19.80 


.000 


% Overall Accuracy 


43.7 


42.4 


26.3 


10.43 


.0001 


1.5 SD Federal Definition 












% Correct LD 


43.8 


40.8 


22.4 


8.57 


, .0003 


% Correct Non-LD 


67.7 


71.8 


53.1 


8.24 


. 0004 


Z Overall Accuracy 


53.6 


53.6 


35.0 


24.85 


. 0000 
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Table 4 

Percentages of Judges Classifying Student as LD (Leniency) 
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Student Classification 





Low Ach 


Federal 


Teachers 


Sch. Psyc. 


Naive 


2 

X (df=2) 


P 


1 


L-L-L 


L-N-N 


26.3 


18.5 


85.7 


33.7 


.0000 


2 


N-L-L 


N-L-L 


15.8 


15.4 


9.5 


.5 


.77 


3 


L-L-N 


L-L-N 


71.1 


61.5 


14.3 


19.2 


.0001 


4 


N-L-N 


N-L-L 


89.5 


90.8 


23.8 


4o.5 


.0000 


5 


L-N-N 


L-L-L 


26.3 


21.5 


14.3 


1.2 


.56 


6 


L-L-N 


L«L-L 


63.2 


67.7 


23.8 


13.0 


.002 


7 


N-L-L 


N-L-L 


63.2 


52.3 


14.3 


13.5 


.001 


Q 
o 


ii n ii 


li— 1-1 li 




1 ft 


■^ft 1 

JO . J. 


O.J- 


it; 


9 


N-L-N 


N-L-L 


21.1 " 


7.7 


23.8 


5.2 


.07 


10 


L-N-N 


L-L-L 


40.5 


30.8 


4.8 


8.4 


.01 


11 


N-L-L 


N-N-N 


34.2 


23.4 


66.7 


13.1 


.001 


12 


L-L-L 


L-L-N 


44.7 


58^ 


90.5 


11.8 


.003 


13 


N-N-N 


N-L-N 


18.4 


12.3 


14.3 


.7 


.70 


14 


L-L-N 


L-L-L 


44.7 


40.0 


9.5 


8.1 


.02 


15 


N-L-N 


N-L-L 


10.5 


17.2 


19.0 


1.1 


.59 


16 


L-N-N 


L-L-N 


5.3 


9.2 


19.0 


3.0 


.23 


17 


L-L-L 


L-L-L 


63.2 


64.6 


81.0 


2.3 


.32 



Overall Leniency 39.1 35.6 32.5 



The two columns indicate the classifications of each student (L=LD, N=Non-LD) 
according to three criteri.^. The "Low Ach" column lists the classifications 

of the student according to the school, the 1.0 SD low achievement definition, 

and the 1.5 SD low achievement definition. The "Federal" column lists the 

classifications of the student according to the school, the 1.0 SD federal 
definition, and the 1.5 SD federal definition. 

(2,121) = 1.2, £ = .52. 
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Table 5 

Average Correlations of Agreement with Group Composite 



Composite Group Decisions 
Teachers Sch. Psyc. Naive F(2,121) p 

Teachers .49 .50 .05 36.98 .0000 

School Psychologists .47 .53 .10 31.35 .0000 

Naive " .04 .08 .62 31.76 .0000 
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^ Table 6 

Judges' Ratings of the Degree to Which Factors 
Influenced Their Decisions 



29 



Factor 


MeAns and Ranges 


of Ratings 






ieacners 


ben. rsyc. 


Naive Overall 


Intellectual 


4.3 


4.2 


3.9 


4 


.2 




(3-5) 


(1-5) 


(3-5) 


(1- 


-f.) 


Academic Achievement 


4.4 


4.2 


3.8 


4 


.2 




(3-5) 


(1-5) 


(3-5) 


(1- 


-5) 


Child's Sex^ 


1.9 


1.9 


1.2 


1 


.8 




(1-5) 


(1-4) 


(1-2) 


(1- 


-5) 


Perceptual-Motor 


3.4 


3.6 


3.2 


3 


.5 




(1-5) 


(1-5) 


(1-5) 


(1- 


-5) 


Adaptive Behavior 


3.?. 


3.0 


3.0 


3 


.1 




(1-5) 


(1-5) 


(1-5) 


(1 


-5) 


Expected- Ac tugl Achievement 
Discrepancy 


4.4 


4.4 


3.4 


4 


.2 


(3-5) 


(1-5) 


(2-5) 


(1 


-5) 


Personality Data 


2.7 


?.8 


3.2 


2 


.9 




(1-4) 


(1-4) 


(2-5) 


(1 


-5) 


Behavioral Recordings 


3.2 


3.0 


3.1 


3 


.0 




(1-5) 


(1-5) 


(1-5) 


(1 


-5) 


Subtest Score Discrepancy^ 


4.1 


3.6 


3.0 


3 


.6 




(1-5) 


(1-5) 


(1-5) 


(1 


-5) 


Ganges are the numbers in 


parentheses. 


Variables were rated on 


a 


scale 


where 1 =;pone, 2 = insign 


ificant, 3 = 


moderate, A = 


= significant 







5 = very significant. 

'These factors were rated differentially across groups of judges: Child's 
Sex [£(2,117) = 5.53, j[> < .01]; Exnected - Actual Achievement Discrepancy 
[£(2,118) = 10.64, £ < .0001]; Subtest Score Discrepancy [E(2,115) = 7.68, 

£ < .001]. 
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table 7 ^ 
Average Mean Differences Between Non-LD and TJ) Students for Each Group of Judges^ 



Teachers 


Avg. Mean 
Difference 


School Psychologists 


Avg. Mean 
Difference 


Naive 


Avg. Mean 
Difference 


PIAT Mathematics 


10.0 


PIAT Mathematics 


12.1 


W-J Reading Aptitude 


13,5 


W-J Mepory 


6.6 


W-J Mathematics Achievement 


8.5 


W-J Verbal Ability 


13,2 


W-J Mathematics Achievement 


6.5 


W-J Verbal Ability 


7.8 


WISC-R Full 


13,1 


PUT Total 


6.0 


W-J Memory 


7.7 


WISC-R Performance 


12,9 


WISC-R Full - PIAT Total 


5.5 


PIAT Total 


7.5 


WISC-R Vocabulary 


12,9 


WISC-R Verbal - WISC-R Performance 5, A 


W-J Knowledge Aptitude ' 


7. A 


W-J Knowledge Aptitude 


12,0 


W-J Verbal Ability 


5.2 


W-J Written Language Aptitude 


7. A 


PIAT Total 


12,0 


PIAT Information 


5.2 


PIAT Passage Comprehension 


7.3 


WISC-R Information 


11,8 


V-J Writ, Lang, Aptitude - 
W-J Writ. Lang. Achievement 


5.1 


V-J Writ. Lang. Aptitude - 
W-J Writ. Lang. Achievement 


7.1 


WISC-R Picture Completion 


11,6 


PIAT Passage Comprehension 


5.1 


PIAT Information 


7.1 


PIAT Information 


11,5 


V-J Written Language Aptitude 


5.1 


W-J Readitig Aptitude 


7.0 


PIAT Passage Comprehension 


11,2 



The table presents the first 11 variables for the three groups of Judges, starting with the one of greatest average mean difference. 
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i 


AcKltvtJM^t 
• 


m 

s 

w 

1 


A 


•a 


rur 

m 

z 
t 

1 

5 t 
«* 


2 

! 

B 


w 
H 

i 


m 

w 

1 8 

J u 

i 2 


u 

J I 


if 


i 

1 

3 


1 3 


M 

t 
*• 

t 
« 




72 


7) 


74 


100 


77 


•1 


74 


19 


12 


•1 77 


2 14 


40 


5 


0 7 


2 


92 


92 


92 


17 


91 


94 


H 


91 


101 


97 


U 77 


2 14 


52 


V 

0 


0 0 


1 


79 


17 


94 


12 


91 


94 


101 


92 


102 


97 


91 97 


3 It 


)7 


1 


1 1 


2 


1) 


101 


112 


9] 


72 


104 


M 


97 


91 


91 


II 11 


4 14 


45 


4 


0 0 


7 


9) 


19 


12 


17 


114 




101 


U 


M 


97 


91 95 


4 11 


47 


1 


0 4 


( 


94 


12 


1) 


14 


100 


II 


95 


90 


94 


92 


84 9] 


t 20 


50 


2 


0 4 


9 


102 


91 


92 


9) 


112 


90 


94 


90 


lU 


99 


94 91 


3 1) 


47 


0 


0 0 


1 


H 


1) 


11 


84 


111 


19 


91 


11 


114 


94 


105 77 


1 it 


4) 


0 


0 0 


2 


119 


101 


U 


99 


* 119 


H 
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M 


no 


105 


90 109 


0 17 


21 


10 


0 ) 
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Tests Administered 



All measures have a mean of 100 and standard deviation of 15 unless 
otherwise indicated. 

APTITUDE MEASURES 

WISC-R 



Full Scale 

Verbal Scale 

Performance Scale 

Information 

Similarities 

Arithmetic 

Vocabulary 

Woodcock- Johnson Cognitive 

Broad Cognitive 

Reading Aptitude 

Math Aptitude 

Knowledge Aptitude 

Written Language & Aptitude 

ACHIEVE14ENT MEASURES 

Woodcock-- Johnson Achievement 

Math 
Reading 



Comprehension 
Picture Completion 
Picture Arrangement 
Block Design 
Object Assembly 
Coding 



Verbal Ability 
Perceptual Speed 
Memory 
Reasoning 



Written Language 
Basic Skills 



Peabody Individual Achievement (PIAT) 



Math 

Reading Recognition 
Passage Comprehension 

Stanford Achievement 



Spelling 
Information 
Total Test 



Math Concepts 



Math Calculation 
PERCEPTUAL MOTOR (Use norms provided) 
Bender Visual-Motor Gestalt Test 

Developmental Test of Visual-Motor Integration (VMI) 
SELF-'CONCEPT (Use norms provided) 

Piers-Harris 
BEHAVIOR RATINGS (Use norms provided) 



Peterson-Quay 



Inadequacy- Immaturity 
Socialized Delinquency 



Conduct Problem 
Personality Problem 
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A. BENDER: Honaacive Daca 







Mean 


Scandard 


Plus /Minus 


Ase Group 


N 


Scores 


Deviacion 


S.D 




5-0 CO 5-5 


81 


13.6 


3.61 


10.0 CO 


17.2 


5-6 CO 5-11 


128 


9.8 


3.72 


6.1 CO 


13.5 


6-0 CO 6-5 


155 


8.4 


4.12 


4.3 CO 


12.5 


6-6 CO 6-11 


180 


6.4 


3.76 


2.6 to 


10.2 


7-0 to 7-5 


156 


4.8 


3.61 


1.2 CO 


8.4 


7-6 CO 7-11 


110 


4.7 


3.34 


X.4 CO 


3.0 


8-0 CO 8-5 


62 


3.7 


3,60 


.1 CO 


7.3 
5.5 


8-6 CO 8-11 


60 


2.5 


3.03 


.0 CO 


9-0 CO 9-5 


65 


1.7 


1.76 


.0 to 


3.5 


9-6 CO 9-11 


49 


1.6 


1.69 


.O'co 


3.3 


10-0 CO 10-5 


27 


1.6 


1.67 


.0 CO 


3.3 


10-6 CO 10-11 


31 


1.5 


2.10 


.0 CO 


3.6 



B. DEVELOPMENTAL TEST OF VISUAL MOTOR INTEGRATION (Vljl) ! NonaaCive DaCa 





Age 


Equivalencs 






Agz Equ 


ivalenCs 


Score 


Male 


Female 




Score 


Male 


Female 


1 


2-10 


2-10 




13 


6-10 


6-7 


2 


3-0 


' 3-0 




14 


7-4 


7-2 


3 


3-2 


3-2 




15 


7-10 


7-11 


4 


4-1 


3-10 




16 


8-7 


8-8 


5 


4-4 


4-1 




17 


9-4 


9-6 


6 


4-6 


4-4 




18 


10-2 


10-3 


7 


4-9 


4-8 


i 


19 


10-11 


11-1 


8 


5-0 


4-11 




20 


11-9 


12-0 


9 


5-3 


5-3 




21 


12-8 


13-0 


10 


5-7 


5-6 




22 


13-9 


13-11 


11 


6-0 


5-10 




23 


14-10 


14-10 


12 


6-5 


6-2 




24 


15-11 


15-9 



C. PIERS-HARRIS SELF-CONCEPT 



There are a cocal of 80 teems. The higher che score, £he higher Che self- 
concepc. InsciCuCe Sample for LD and Non-LD (N « 99) X * 52.14, SD - 14.43 
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PETERSON-QUAY BEHAVIOR PROBLEM CHECKLIST: 
Various Samples 



Means and Scandard Deviacions of 



Conduce 



Personality ImmaCuricy 



SD 



SD 



SD 



Public School (r:.ced 
by ceachers): 



43 



Socialized 
Delinquency 

X SD 



K males 


336 


4.52 


4.93 


3.66 


3.56 


1.63 


1.87 


0.34 


0.97 




342 


3.73 


4.4A 


3.68 


3.19 


1.67 


1.81 


0.27 


0.74 


K females 


206 


2.33 


3.76 


3.96 


3.60 


1.16 


1.65 


0.17 


0.6?. 




281 


1.96 


3.09 


3.37 


3.00 


1.26 


1.12 


0.11 


0.44 


Isc males 


309 


4.05 


4.36 


2.80 


3.11 


1.50 


1.69 


0.23 


0.66 




279 


4.24 


4.96 


3.47 


3.5; 


1.62 


2.00 


0.43 


1.15 


Isc females 


257 


1.72 


2.96 


2.62 


2.84 


0.84 


0.13 


O.ii 


0.38 




276 


2.18 


3.3A 


3.20 


2.91 


1.21 


1.71 


0.20 


0.62 


2nd m&les 


294 


4.20 


4.64 


3.01 


3.09 


1.76 


1.88 


0.34 


0.88 




273 


4.12 


4.76 


2.89 


3.19 


1.61 


1.79 


0.43 


1.07 




297 


1.95 


3.30 


2.71 


3.16 


1.09 


1.62 


oh9 


0.58 




276 


2.00 


3.33 


3.30 


3.26 


1.02 


1.54 


0.21 


0.71 


6th male . 


62 


2.93 


3.66 


2.77 


3.13 


1.35 


1.99 






6ch female 


41 


1.44 


2.45 


2.76 


3.30 


1.02 


1.63 







ID 0 



Response Fonn 



After examining the test scores for each child « indicate your decision of 
LB or non-LD, Then indicate how certain you are of your decision. Please 
use the back of this page to record any comments you have concerning the 
sorting task you have completed* 
)b 

Subject Very Very 

Kuaber LP Non-LD Certain Certain Uncertain Uncertain 



1 
2 
3 




10 
11 
12 
13 
lA 
15 
16 
17 
18 



Please rate the extent to which each of the factors listed below affected 
your decisions. Use the following numbers to indicate the effect of each 
factor* 

1 • None 2 - Insignificant 3 - Moderate A - Significant 5 - Very Significant 

Scores on intellectual measures 
. Scores on measures of academic achievement 

The child's sex 
. Scores on perceptual-motor tests 

Adaptive behavior 

Llscrepancles between expected and actual achievement 

Personality test data 

Behavioral recordings 

Subtest score discrepancies ^ 

Other (please describe) 
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